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Executive summary 


The project 


The Teacher Effectiveness Enhancement Programme (TEEP) is a CPD programme that aims to 
improve teachers’ classroom practice. TEEP training is offered as a whole-school approach by the 
Schools, Students and Teachers Network (SSAT). All staff in a school received three days of training 
over a period of two terms. A smaller cohort of teachers, chosen by the school leadership team, receive 
two further days of training the following term to help embed and develop the TEEP strategies across 
the school. The training focuses on developing pedagogical knowledge (for example, assessment for 
learning or collaborative learning), understanding different phases of learning (such as preparing for 
learning and demonstrating understanding), and effective teacher behaviours (including classroom 
management and interactive teaching). The programme provides a comprehensive language and 
framework for understanding how to improve the effectiveness of both teachers and learners in 
secondary schools. 


TEEP was evaluated using a randomised controlled trial with over 10,000 students in 45 low-performing 
secondary schools across England. Schools were eligible for inclusion in the project if their performance 
was below government floor standards at the point of recruitment. Schools were randomly allocated to 
receive either the TEEP training or continue with ‘business as usual’. The primary outcome measures 
were grades in GCSE English and mathematics sat by pupils in either 2014 or 2015, at least 18 months 
after the year-long TEEP programme began. The secondary outcome measures were scores on the 
CEM INSIGHT tests—which measure science, maths and reading attainment—and attitudes to school 
at Year 9, but this data was not collected as intended. The evaluators also attended the TEEP training, 
interviews with senior staff and teachers, and focus groups and surveys as part of the process 
evaluation. They also conducted lesson observations. 


Key conclusions 


1. In the low-performing schools selected for this trial, there was no evidence of an impact on 
pupils’ GCSE English and maths attainment in schools selected to receive TEEP training 
compared to other schools. 


2. Both teachers and students were enthusiastic about the programme and believed that it 
improved students’ learning. 


3. The evaluation was initially designed to also assess implementation quality and the impact 
on Year 9 attainment and attitudes. However, the relevant measures for these key evaluation 
elements were not collected due to circumstances beyond the control of the project teams. 
This means the evaluation cannot assess whether TEEP made a difference to Year 9 pupils, 
as originally intended. 


4. Any future studies could systematically investigate the implementation of TEEP using 
implementation measures that relate to specific components of the programme, as well as 
investigating changes in school culture and teacher behaviour. 


er Security rating awarded as 
How secure are the findings ? rt of the EEF ria 


The findings from this evaluation for the primary outcomes are judged to be of moderate security. It 
should be considered an effectiveness trial as it aimed to test a scalable intervention under realistic 
conditions in a large number of schools. The trial used a well-conducted experimental design and 
appropriate analysis. At the beginning of the trial the schools and pupils who received the intervention 
were similar to the schools and pupils in the comparison group. Two padlocks are removed from the 
rating because over 20% of pupils’ GCSE results were not included because their initial attainment data 
at age 11, which is needed for the analysis, was not available. 
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What are the findings? 


The evaluation of the TEEP programme showed no effect on GCSE English or maths scores in the low- 
performing schools that participated in the trial, nor did it have an impact for students eligible for free 
school meals, for either gender, or for students of different ability levels. 


Due to circumstances beyond the control of the evaluator and developer, Year 9 attainment and attitudes 
data, and data on the quality of implementation, were not collected as planned. For these reasons it is 
impossible to determine the impact of the intervention on Year 9 attainment or attitudes compared with 
the control group, or to assess whether problems with implementation contributed to the lack of positive 
impact. 


Interviews and focus groups indicated that most of the teachers in the TEEP schools found the training 
useful and were keen to implement TEEP methods and principles in their lessons. Survey data indicated 
that teachers and students believed that TEEP had made them more effective teachers and learners 
overall. Survey responses suggested that teachers and pupils felt TEEP had less of an impact on 
effective use of ICT than on other elements of teaching, learning and classroom activity. 


Previous evaluations of TEEP suggested that the programme had an impact on the behaviour and 
perceptions of pupils and teachers, but this is the first randomised controlled trial examining the impact 
on attainment. Future studies could systematically investigate the implementation of TEEP using 
implementation measures that relate to specific components of the programme, as well as investigating 
changes in school culture and teacher behaviour. 


How much does it cost? 


The cost of TEEP training is based on the number of teachers and students in the school. For a school 
with up to 50 staff members and 1,000 students SSAT would charge approximately £13,000. The cost, 
therefore, spread over three years is approximately £4,333.00 per school per year and the cost per 
student per year is £4.33. 


Table 1: Summary of impact on primary outcomes 


Effect size j 
Group & Number of SSO 25% EEF cost 


(95% confidence months’ | security 
interval) progress rating 


Outcome Ted aTefo) f rating 


Treatment vs. 
control on 
English 
Treatment vs. 
control on 
Maths 
Treatment FSM 
vs. control -0.03 

FSM on (-0.16; 0.09) 
English 
Treatment FSM 
vs. control 
FSM on Maths 


-0.04 
(-0.17; 0.10) 


-0.02 
(-0.13; 0.10) 


-0.01 
(-0.14; 0.12) 


1 Since this report was published, the conversion from effect size into months of additional progress has been 
slightly revised. If these results were reported using the new conversion, all measures would be reported as O 
months of additional progress rather than -1. See here for more details. 
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Introduction 


Intervention 


The Teacher Effectiveness Enhancement Programme (TEEP) is a continuing professional development 
(CPD) programme for teachers initially developed in 2002 through funding from the Gatsby Charitable 
Foundation. TEEP aims to improve teachers’ classroom practice by enabling them to reflect on their 
current practice, align this to a pedagogical framework, and to implement teaching strategies that nurture 
and develop the characteristics of effective learning in their students. The TEEP programme is neither 
subject- nor age-specific and consists of a framework model and training that are designed to enable 
teachers to become more effective and thereby improve learners’ effectiveness. 


TEEP translates both classroom practice and educational research into a teaching and learning 
framework and training programme designed to impact long term pupil achievement through a positive 
shift in culture for both teachers and learners. TEEP was designed to influence teacher behaviours, both 
professional characteristics and patterns of behaviour, which in turn should improve the overall 
standards of teaching and learning, develop a culture of learning in teachers and learners, and provide 
a consistent approach of high expectations, appropriate challenge, and formative assessment that 
creates an ethos of high expectation and high achievement. 


The TEEP model 


Represented as a graphic (see Figure 1 below), the TEEP model or pedagogical framework is broken 
down into three tiers; 


e asix-phase learning cycle (preparing for learning, agreeing learning outcomes, presenting 
new information, constructing meaning, demonstrating understanding, and reviewing); 

e five underpinning elements (assessment for learning, thinking for learning, accelerated 
learning, collaborative learning, and effective use of ICT); and 

e effective teacher and learner behaviours (interactive teaching, classroom management, variety 
of teaching approaches, and classroom climate). 


Aspects of these three tiers are considered in more detail below. 


Figure 1: The TEEP model describes a taxonomy of thinking for teachers. 
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Effective teacher behaviours that TEEP promotes focus on using a variety of teaching and learning 
approaches: interactive teaching, classroom management, and classroom climate—including high 
expectations and strong relationships. Effective learner behaviours include making decisions, seeking 
assistance, checking personal progress, asking questions, making links to previous learning, developing 
strategies for learning, and reflecting on learning. 


TEEP delivery model 


TEEP training is offered as a whole-school approach with a five day training programme. The training is 
delivered by SSAT trainers who have classroom experience and who have been rigorously trained in 
TEEP. From 2012, following the previous evaluations, SSAT developed the whole-school TEEP training 
to make it more suitable for EEF target schools, and more focused on raising the attainment of the most 
disadvantaged pupils. 


Whole-school training involves all teachers, leadership, and high level teaching assistants who are 
involved in the planning, delivery and assessment of learning. This is a key aspect of SSAT TEEP: 
schools about to embark on their TEEP journey will commit to ensuring all members of staff are included 
in level 1 training without exception. The developers have found that schools where senior leaders do 
not engage fully with TEEP level 1 will struggle to maintain the enthusiasm following training. They 
believe that successful schools have the clear identification of the headteacher as lead pedagogue. 


The five days of training are broken down as follows: 


e Level 1: all teachers including leadership and support staff—three days of training (two 
consecutive days with a gap of approximately six to eight weeks before day three); then 

e Level 2: smaller cohort of teachers, from those who participated in Level 1, chosen by the 
school’s senior leadership team to participate in an additional two days of training 
(approximately 8—12 weeks after day three). 


All days involve six hours of face-to-face training which is participatory, active, and aligned to school 
context. The delivery of TEEP varied from school to school depending on the existing practices and 
experience of the teachers and the composition of the student body. 


Level 1 (taken from the TEEP manual) 


Day 1: Building the classroom climate, the culture of learning and the five underpinning elements 
(see Figure 1 above): 
e assessment for learning; 


e thinking for learning; 
e accelerated learning; 
e collaborative learning; and 
e = effective use of ICT. 


Teachers will apply the principles of the underpinning elements when planning learning activity such 
that learners are exposed to higher-order thinking skills, formative assessment techniques, 
collaboration, co-construction and so on. 


Day 2: The TEEP learning cycle (see Figure 1 above). 


This concerns the phases of learning a pupil experiences in a lesson or a series of lessons. Teachers 
will understand the six phases of learning and how this informs effective planning and intervention in the 
classroom: 


e preparing for learning; 
e agreeing learning outcomes; 
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e presenting new information; 

e constructing meaning; 

e demonstrating understanding; and 
e = reviewing. 


Teachers are able to move back and forth throughout the cycle depending on learner need. 
Day 3: Effective teacher behaviours and effective learner behaviours. 


Effective teacher behaviours are the very foundations of the TEEP framework and focus on: 
e interactive teaching; 
e classroom management; 
e variety of teaching approaches; and 
e classroom climate. 


Level 2 


Days 4 and 5: coaching and mentoring. 


The members of staff who have been chosen by their school’s senior leadership team to lead the 
implementation of TEEP in their school receive two additional consecutive days’ training in coaching, 
co-coaching, and mentoring aligned specifically to the TEEP model. 


During training, teachers will experience: 


e mentoring and coaching skills (such as demonstrating the skills of active listening and 
paraphrasing and collaborating with colleagues to complete the creation of a ‘TEEP teachers’ 
toolkit’); 

e participating in lesson observations (by, for example, observing a TEEP lesson and discussing 
how to give feedback in a non-judgmental way, understanding the TEEP coaching framework); 
and 

e conducting classroom observations (by understanding and applying the three purposes of 
classroom observation and observing effective and appropriate questions in the classroom). 


TEEP leadership training 


The project also involved further support through additional leadership training for all schools, including 
a headteachers’ briefing before, and senior leadership training after, the core training, as well as ongoing 
support. 


This stage of the programme was implemented as it is key that school leadership teams need to (a) lead 
by example, (b) understand the need for a coherent vision, and (c) fully support staff delivering the 
programme if it is going to be successfully embedded in a school. Issues such as changes in leadership, 
both of the school and/or the Academy Trust, plus high turnover of staff would lessen the ability of TEEP 
to be consistently applied across school and therefore limit the impact. 


The headteachers’ briefing prior to Level 1 training brings together the headteachers of that year’s 
schools and provides the opportunity to: 


e understand the history of TEEP and the research evidence behind it; 

e define the importance of a clear vision for teaching and learning in school before TEEP starts; 

e support the planning for the embedding and sustaining of TEEP across the school/academy; 

e provide clear tools and information to introduce TEEP to the school community (staff, parents, 
governors, and students); and 
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e understand how, through TEEP, staff can be supported to improve teaching and learning. 


The senior leadership training day—after the Level 2 training for most schools—provides the opportunity 
for the key staff of each school to come together to: 


e identify the success criteria for embedding the TEEP model in classroom, subject disciplines, 
departments, and school; 

e compare and contrast CPD strategies for embedding TEEP, exemplified in TEEP-trained 
schools, and evaluate these to ensure the impact of the programme is sustained; 

e evaluate and develop their school teaching and learning strategy to support effective teaching 
and learning through application of the TEEP model; and 

e to support the schools involved in the EEF project to ensure that TEEP will be as effective as 
possible for the young people within school. 


Other support for schools or TEEP individuals 


Resources to support TEEP are provided as part of the training and include TEEP booklet sets, A3 
summary mats, thinking posters, and so on. There is a website offering many resources such as 
activities, PowerPoint presentations, and the opportunity to share resources with colleagues. There is 
also a newsletter with articles based on TEEP in practice from practitioners across the UK. 


Background evidence 


TEEP has been evaluated previously, though not in the form evaluated in this project and not through 
the use of an RCT. The four previous qualitative evaluations were commissioned by the Gatsby 
Charitable Foundation. These evaluations collected data on teacher behaviours and perceptions but did 
not collect student achievement data. They aimed to provide independent recommendations to guide 
further development of the intervention. Two pilot evaluations involving seven and then five schools 
were conducted by Gunraj (2005, 2010). Serret and Reiss (2006) conducted a three-year evaluation of 
four schools which used TEEP to support their science teaching. 


A comprehensive qualitative evaluation was carried out of TEEP Phase 2, which was the second version 
of the programme. The evaluation was conducted between 2001 and 2008 by a team from the 
Department of Educational Studies at the University of York (Ragbir-Day, Braund, Bennett, and 
Campbell, 2008). The aims of the evaluation were to identify (a) the factors that resulted in schools and 
teachers becoming involved in TEEP, (b) impact on classroom practice, (c) impact on pupils’ learning 
experiences, and (d) factors impacting on ‘ownership of TEEP and the ways in which teachers chose to 
use TEEP’. 


Data was gathered from two primary schools, 11 secondary schools, and two further education (FE) 
colleges. The data was gathered via (a) teacher observations, (b) pupil questionnaires, (c) teacher 
interviews, and (d) interviews with key informants. Rather than use examination or test results, the pupil 
data was drawn from 582 pre-training questionnaires and 520 post-training questionnaires. The findings 
of the evaluation were mixed, though there were more positive behaviours of teachers than negative 
ones. In keeping with the TEEP approach, positive impacts included the following changes in teacher 
behaviour: 


e less time spent controlling pupil behaviour (all institutions); 

e less whole-class teaching (Secondary, FE); 

e increased use of group work (Secondary, and in one primary and one FE institution); 
e more inclusive questioning (secondary, FE); and 

e increased use of ICT (Secondary, FE). 
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The research team reported that they found increased application of all five of the underpinning elements 
of TEEP, but also stressed that within these overall patterns there were considerable variations among 
institutions, as outlined above. 


Two of the most important recommendations from this study were, first, that ‘a critical mass of TEEP- 
trained teachers is needed for consistent application of TEEP approaches (all institutions)’, and second, 
that there was a need for ‘time for practice to become embedded, with noticeable benefits being 
apparent after two years (all institutions)’ (Ragbir-Day et al., 2008). 


All of the studies above provided recommendations for further development and implementation. They 
suggested that there was sufficient positive feedback about the TEEP approach to warrant a large-scale 
evaluation. They concluded that, as TEEP has been scaled up and was being implemented in schools 
around the country, an effectiveness trial was needed. 


Evaluation objectives 


The impact evaluation for this project was intended to be an RCT involving 40 secondary schools to 
assess the effects of TEEP, particularly on raising the attainment of the most disadvantaged secondary 
school students. (To view the evaluation protocol click here: 
https://educationendowmentfoundation.org.uk/public/files/Projects/EEF_Project_Protocol_Tea 

cherEffectienessEnhancementProgramme.pdf). 


The process evaluation aimed to assess the quality of implementation of TEEP in a small number of 
schools. The goal of the information gathered was to inform SSAT about the factors that impact 
implementation and help guide the programme’s future design. Neither the evaluation team nor SSAT 
had any structured monitoring of implementation fidelity which would have enabled relating it to student 
attainment. 


Ethical review 


The evaluation proposal was submitted to the Department of Education (University of York) Ethics 
Committee. Ethical approval was granted in July 2012. Information and consent forms were issued to 
headteachers and letters of assent to parents. 


Project team 

Anne-Marie Duguid, Director of Education, SSAT. 

Andy Williams, Senior Educational Lead, Teaching and Learning, SSAT. 
Corinne Settle, Educational Leads, Teaching and Learning, SSAT. 
Sharon Simpson, Educational Leads, Teaching and Learning, SSAT. 
Fie Rason, Senior Project Manager, SSAT. 

This list does not include the qualified TEEP trainers who delivered the school courses. 
Evaluation team members 

Mary Sheard, Principal Investigator (deceased). 

Peter Rudd, Principal Investigator. 

Alaidde Berenice Villanueva Aguilera, Research Associate. 


Louise Elliott, Data Manager. 
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Bette Chambers, Co-Investigator. 
Robert Slavin, Co-Investigator. 


Chris Whitaker, Statistical Consultant. 


Trial registration 


As one of the first EEF trials—and prior to the registration of the EEF—this trial was not registered. 
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Methods 


Trial design 


This study of TEEP builds on previous evaluations by employing an effectiveness evaluation using a 
two-armed clustered randomised controlled trial. Randomisation was at the school level and carried out 
over two separate cohorts. School-level randomisation was chosen because the intervention is a whole- 
school intervention. 


To achieve sufficient power, TEEP was implemented in a large number of secondary schools and no 
primary schools. There was a waitlist control group, an ideal counterfactual for the TEEP group. The 
control schools volunteered to continue with business as usual and implement the programme (at no 
cost to the school) after the evaluation was finished. Matched schools randomisation was used to ensure 
that TEEP and control schools would be very similar across all relevant factors, such as prior 
achievement and the percentage of students on free school meals (‘FSM students’). 


Although TEEP is a whole-school model, the evaluation focused on the attainment of the students in 
one year level—those who would take their GCSEs in June 2014 or June 2015—compared to an 
equivalent control sample. Each cohort of students was exposed to TEEP for one and a half to two years 
prior to taking those GCSEs. 


The schools that participated in the evaluation were located all around England. The length of 
implementation varied depending on when the schools were trained and began implementation. 
Teachers from the first schools were not trained until late autumn 2012 and then the students took their 
GCSEs in June of the following school year (after approximately 18 months’ exposure to TEEP). Schools 
in the second cohort started training earlier in the school year of 2013 and took their GCSEs in June of 
the following school year (after about two years of implementation). 


The impact evaluation included analyses of GCSE results, using KS2 scores as a covariate, to 
investigate whether implementing the TEEP programme in secondary schools led to improvement in 
students’ test scores. Researchers also conducted a process evaluation, attended TEEP training, and 
visited a number of schools during the implementation period. 


Participant selection 


Eligibility was decided first by the EEF and focused on secondary schools performing around or below 
the government floor targets at the time for GCSE performance. In other words, struggling schools were 
targeted—those with a lower percentage of pupils achieving five A*-C GCSEs—rather than well- 
performing schools. Recruitment was conducted by SSAT through invitation to approximately 1,000 
such schools. 


Recruitment followed these steps: 


1. Recruitment information was sent to schools and publicised widely through SSAT networks. 

2. Expressions of Interest were received from schools by completing a form online by a specific 
deadline. 

3. Applications were longlisted using the criteria above, based on the information provided in the 
application. 

4. All schools were offered a telephone interview with a member of the educational team to 
discuss the programme, the school’s suitability, and the commitment (Some schools never 
returned the phone call so excluded themselves at this stage). 

5. All schools who had been involved in the telephone interviews were reviewed again and a 
shortlist was created based on the information provided by each school regarding its current 
position, its teaching and learning, and the ability and willingness of the senior leadership 
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team to commit to the requirements of the project, including partially funding the training 
(around £3,000—£5,000 per school). 

6. The shortlist was shared with the EEF to check it was satisfied with the selected schools. 

7. The shortlisted schools were sent a copy of the ‘prerequisites’ to sign, confirming acceptance 
of the conditions of programme. This included that it could be allocated to either treatment or 
control year group. Any schools that did not return a signed form were removed and replaced. 

8. A final list of schools was approved and given to the IEE to randomise. 

9. The Unique Pupil Numbers (UPNs) were collected from the schools by the IEE researchers 
after schools were selected. 


Due to the project being a whole-school intervention and with the increase in the number of schools 
requested by both the EEF and the evaluation team to ensure a large enough sample for sufficient 
power, the decision was made to run the project over a four-year period in two cohorts. All four years of 
schools (intervention and control) were offered the treatment by the end of the project. 


Headteachers signed a memorandum of understanding before participating in the study (see Appendix 
C). All of the teachers in the schools were expected to participate in the training and implementation of 
TEEP. Parents or carers of children in the classes that were to participate in the study received 
information sheets and opt-out forms (see Appendix D). 


Outcome measures 


The primary post-test measures were routinely collected GCSE scores for both cohorts. These 
measures are high in contextual validity as they constitute the main indicators of school and student 
academic performance. Because virtually all schools in England use these tests, it was assumed that 
control group teachers, as well as TEEP teachers, would be focused on ensuring that students 
succeeded on them. 


Analyses focused on GCSE performance in English and maths (key indicators of academic success and 
therefore the focus of EEF evaluations), controlling for the individual students’ KS2 scores as provided 
by the NPD. Students’ UPNs were provided by the secondary schools. 


The protocol states that in the analyses of the post-test GCSE scores, schools’ previous GCSE scores 
(those sat in 2012) would be used as covariates. However, at a meeting at the EEF on 21 July 2015, it 
was decided to use individual students’ KS2 scores as covariates instead. This was necessary as, owing 
to the length of the two-cohort trial, there were marked changes in some schools—such as some 
becoming academies. In these circumstances, it was felt that individual students’ KS2 scores would 
constitute a stronger design than school-level GCSEs. Using individual-level KS2 scores allows for a 
more precise estimate and increases the power of the analysis because it relates directly to each 
individual student's prior attainment, rather than a previous population of the school. 


Subgroup analyses were conducted to assess the impact of TEEP on GCSE scores for students eligible 
for FSM, boys and girls, and for high, average, and low achievers. 


It was planned to have students complete the CEM INSIGHT tests as secondary post-test measures to 
capture the effect of TEEP on students in the year levels before GCSEs are taken, however there were 
unavoidable problems with the external testing and communications around these. Testing was carried 
out with the two cohorts of schools. In June 2014 only five schools from a possible 18 completed tests, 
and in June 2015 only 14 schools out of a possible 27 completed some parts of the tests. This represents 
considerable missing data. 


There are a number of factors behind these difficulties, including the death of the PI in July 2014. A 
change in the testing plans from a 50-minute MidYIS test to three 45-minute INSIGHT tests (one for 
each of reading, maths, and science) proved to be too onerous for schools. Also, communication 
difficulties between IEE and the testing organisation (CEM) hindered administration of the tests. At the 
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July 2015 meeting it was agreed that, as was originally planned, the analysis would be of GCSE data 
as the main outcome measure and to not analyse the post-test INSIGHT data. While it is unfortunate 
that the CEM tests were not administered, the GCSE tests were always intended to be the primary 
outcome measure. 


Sample size 


A collaborative decision was taken early on to recruit 52 secondary schools (26 experimental, 26 control) 
to participate in the study to provide more statistical power. 


The statistical power of the planned analyses for the secondary schools was estimated using Optimal 
Design software. With 100 students per school and 52 schools, power of +0.80, intraclass correlation of 
0.15, and p < 0.05, the minimum detectable effect size was +0.20 for the analysis of the outcome data 
with a 95% confidence interval. We estimated the pre-post correlations to be +0.60. 


There were 45 schools in the final ITT analysis (See Figure 2: participant flow diagram). After 
randomisation and implementation began, another school was closed and a further two schools dropped 
out leaving 42 schools for the on-treatment analyses. 


Randomisation 


Randomisation was conducted by the IEE data manager. Schools were matched in pairs using the 
following criteria: average percentage GCSE (A*—C) results over three years, percentage of FSM 
students, percentage of EAL (English as an additional language) students, and the number of students 
registered at the school. Then, using a random number generator, matched schools were assigned to 
be a treatment or a control school. 


Analysis 


Primary and subgroup analyses were conducted with SPSS on an ‘intention-to-treat’ basis. We also 
conducted an on-treatment analysis on the 42 schools that remained in the study to determine if the 
three schools that dropped out of the study made a difference to the results. 


The assessment data for Cohort 1 and Cohort 2 were compared to each other to determine if there were 
differences and to test for equivalence (see Appendix J). No significant differences were found so the 
data from the cohorts was combined and analysed together, providing a more statistically robust 
analysis. 


To account for the clustering and matching in the design a multilevel model was used. The random 
effects were Pair and School nested within Pair. The Basic model used only included Treatment as a 
fixed factor. The covariate model extended the Basic model by including covariates. The covariates for 
the GCSE English and Maths scores were the corresponding baseline scores from KS2. 


We analysed the data for the following subgroups: Gender, Cohort, ‘Ever received Free School Meals’, 
and three ability groups. The model only included random effects described above together with 
Treatment. The ‘high’, ‘low’ and ‘average’ achievers were defined as follows. High performers were in 
the upper quartile of English or maths and, if not in the upper quartile for both, were in the upper for one 
and the middle quartile of the other subject. Low performers were in the lower quartile of English or 
maths and, if not in the lower quartile for both, were in the lower quartile for one and the middle quartile 
of the other subject. The remaining ‘average’ pupils, who mainly scored between the upper and lower 
quartiles for both English and Maths, were assigned to the middle quartiles attainment group. This 
resulted in similar numbers in each of the three groups (See Table 2 below). These groups were used 
for all the outcome analyses. 
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Table 2: Assignment to attainment groups 


Lower quartile Middle quartiles sare Total 

Lower quartile 1,773 969 70 2,812 

: Middle quartiles 883 3,430 1,119 5,432 
Upper quartile 51 1,080 1,518 2,649 
Total 2,707 5,479 2,707 10,893 


This produced: 


Low attainers having ‘neither maths nor English in top 25%’: 1,773 + 969 + 883 = 3,625 pupils; 

High attainers having ‘neither maths or English in bottom 25%’: 1,080 + 1,119 + 1,518 = 3,717 pupils; 
and 

Middle attainers being the remainder: 51 + 3,430 + 70 = 3,551 pupils. 


We had planned to calculate the correlation between level of implementation fidelity in a school and the 
student GCSE attainment, however this was not possible as there were no consistent objective 
assessments of implementation fidelity conducted. 


All 45 schools were included in the ITT analyses. On-treatment analyses were also performed on the 
three GCSE scores with the KS2 scores as the covariate of 42 schools. One intervention and one control 
school withdrew from the study and one school closed so these three schools were excluded, leaving 
42 schools for the on-treatment analyses. 


Implementation and process evaluation 


Since the Teacher Effectiveness Enhancement Programme is a whole-school intervention, the 
evaluation team devised a process evaluation methodology that would examine the perceived impact 
of TEEP from a number of different angles, including trainer, school leader, teacher, and pupil 
perspectives. This also allowed us to look at some of the more subjective aspects of TEEP and not just 
at GCSE examination results as the outcome. 


Building on previous evaluations of TEEP, the process evaluation examined how the project was 
implemented on the ground, and a central question was ‘what are the mechanisms for change?’. This 
aspect of the evaluation included assessment of the quality of implementation of TEEP elements in a 
small number of schools. The process evaluation provided further information about the factors that 
have been perceived to influence the implementation of TEEP (for example, the lack of implementation 
of the ICT component), and this should assist with refining the design of the programme for future 
implementations. A rigorous implementation fidelity measure was intended to capture implementation 
across all the schools, but unfortunately this was not conducted. 


Specific questions used in the observations, focus groups, interviews, and surveys were as follows: 


e How, if at all, has TEEP improved teachers’ classroom practice? 

e How, if at all, has TEEP improved learners’ effectiveness? 

e What have been the challenges in implementing the different elements of TEEP? 
e How consistently have the five TEEP strands been implemented? 

e What additional support might be needed to make TEEP work well? 

e How can the long-term impact of TEEP be sustained? 


In the original proposal we planned to visit 12 randomly chosen intervention schools for the process 
evaluation. Since fewer than expected schools were recruited in the early stages of the project, and the 
timeline for implementation was changed, we revised this so that we visited fewer schools in total, but 


Education Endowment Foundation 15 


The Teacher Effectiveness Enhancement Programme 


more frequently and in more depth. There was also more of emphasis on finding out what was happening 
longitudinally in intervention schools. The numbers of visits, interviews, and focus groups carried out 
are summarised in Table 3 below. 


Table 3: Overview of the process evaluation 


Overview of process evaluation: numbers and details of school visits and interviews 


Numbers and types of school visits 


Two researchers visited or conducted telephone 
interviews with 14 intervention schools before training. 


ited a ae They conducted interviews and obtained information 
about the school pre-TEEP, and asked what the school 
hoped to gain from TEEP. 
Training 2 training Two researchers visited one school each. Each 


observations 


Discussions 
with teachers 
during training 


Control school 
visits (for 
comparisons) 


Post-training 


sessions 


12 teachers 


2 school visits: 


+7 telephone 
interviews 


10 school visits 


researcher took part in the full TEEP introductory training 
course. 


During two three-day training sessions informal 
discussions were held with teachers to obtain their views 
on the training (a total of 12 teachers in two schools). 


Four comparison schools were featured. Two of these 
were visited by researchers (interviews with heads / 
principals, heads of department, teachers) and where 
possible, lesson observations. 

Seven further contacts were by telephone (head 
teachers, deputy heads or department heads 
interviewed). 


Three researchers visited 10 schools in total: interviews 
with heads / principals, TEEP leads, heads of 


school visits department, teachers, and where possible, lesson 
observations. 
NUMBERS AND TYPES OF INTERVIEWS WITH SCHOOLS 
Headteachers/ 


Assistant Heads 


Individual, face- : ; 

20 interviews, 10 schools 
Individual, face- 

24 interviews, 10 schools 


TEEP Leads 
Heads of : 
Interviews and : . 
Teachers 19 teachers in 10 schools (usually with HoDs) 
hascon 1-3 observations 7 
Obearvations for oo 16 observations in 8 implementation school visits 
Control Individual, or 2 face-to-face interviews with headteachers and 10 
teachers/ telephone telephone interviews with heads of departments in a total 
headteachers interviews of 12 control schools. 
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Lesson observations 


Lesson observations were structured around the key TEEP components. Researchers conducted 16 
observations, mostly in English, mathematics and science lessons in a random sample of eight of the 
implementation schools randomly chosen for the process evaluation. 


A lesson observation schedule was devised that enabled the researcher to assess the strength and 
depth of the use of the planning cycle, the five strands, and the teaching and learning behaviours that 
teachers and students engaged in within a lesson (See Appendix E for the TEEP Lesson Observation 
Schedule). 


Interviews 


Semi-structured interviews were conducted with the TEEP headteachers or lead teachers in each of the 
eight schools visited during the school visits (see Appendix F for the interview questions). These 
interviews were to elicit the school’s priorities and its perception of the impact of TEEP on teaching and 
learning. 


The two visits and two telephone interviews with the control schools were conducted to see if the control 
schools were focusing on improving teaching and learning in their schools in the same way that the 
TEEP programme was aiming to bring about improvements. 


Focus groups 


Focus group meetings were held with teachers of various subjects, trained in TEEP, from a random 
sample of ten of the implementation schools to assess school priorities and predictions of how TEEP 
would impact teaching and learning (See Appendix G for the focus group protocol). These focus groups 
were conducted alongside the lesson observations. 


Surveys 


Online surveys were sent to participating teachers in the implementation schools to obtain their 
perceptions about whether and how TEEP had influenced their teaching (see Appendix H for the teacher 
survey). 


Student surveys were made available online to the treatment schools to administer to the students to 
assess how their learning had changed and whether they had they noticed difference in respect to the 
five components of the TEEP programme (see Appendix | for the student survey). 


Analysis 


Descriptive statistics were computed for the teacher and student surveys from the intervention schools 
to assess their perceptions of the TEEP programme and how it affected their teaching and learning, 
particularly the five TEEP components. 


Content analyses were conducted on the lesson observation, interview, and focus group data collected 
from the headteachers, TEEP schools leads, teachers, and students in the intervention schools. Again, 
the interviews and focus groups concentrated on the participants’ perceptions of the influence of the 
TEEP programme. Triangulating among the various measures, we looked for patterns of change in 
attitudes, priorities, and behaviours. 


Unfortunately, Mary Sheard, the principal investigator of the evaluation, died suddenly in July 2014. This 
loss of the PI made analysing the process evaluation data very challenging. Mary had established good 
relationships with the participating schools and it was difficult to replicate her efforts in the time remaining 
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for the project. Her hand-written notes on the interviews and lesson observations were very difficult to 
read. 


Costs 


Because TEEP is currently being implemented across the country, we calculated the costs of TEEP by 
asking SSAT how much they charge an average school to implement the programme. As these are 
upfront costs, we then divided those costs by three to get a cost per year (on the assumption that a 
trained school would continue with the approach for three years) and then by the average number of 
students in an average school. 


All of the teachers in each intervention school spent three days attending training. The TEEP mentors 
in each school also participated in two days of additional training. Because TEEP is a whole-school 
model and is not an additional intervention but replaces normal in-class teaching, we did not include 
staff time in our cost calculations. It did take teachers additional time to prepare lessons according to 
TEEP while they were mastering the approach, but we have no idea how much additional time that took. 


Timeline 


The timeline for completion of the final report was extended for a number of reasons including the 
necessity of having two cohorts and the loss of staff from the evaluation team. See Table 4 below for a 
detailed timeline. 
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Table 4: Timeline of work performed 


School Year 2012/2013 School Year 2013/2014 School Year 2014/2015 School Year 
2015/2016 
Summer | Autumn | Spring | Summer Autumn Spring Summer Autumn Spring | Summer 
Term Term Term Term Term Term Term Term Term 
Randomisation 
by IEE 
Implementation Control Cohort 
Cohort 1 i nn a> al 
Implementation Experimental Control Cohort 2 
Cohort 2 eu 
Lesson Control Cohort Experimental Control Experimental 
observations 1 Cohort 2 Cohort 2. | Cohort 2 
Experimental 
Cohort 2 
Focus groups Control Cohort Experimental | Control | Experimental 
1 Cohort 2 Cohort 2 | Cohort 2 
Experimental 
Cohort 2 
Electronic Experim | Experimental 
surveys ental Cohort 2 
Experimental Cohort 2 
Cohort 2 
Conduct Experimental Cohorts 1 and 2 
Analyses & Control 
Cohort1 
Experimental 
& Control 
Cohort2 
Submit final May 2016 
report 
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Impact evaluation 


Participants 


The evaluation was planned to be conducted over two cohorts, Cohort 1 beginning in 2012/2013 and 
Cohort 2 in 2013/2014. Cohort 1 consisted of 18 schools. For Cohort 2, 29 schools were recruited but 
two dropped out leaving 27, making a total of 45 schools for the intention-to-treat analyses (See Figure 
2 for a Participant Flow Chart for the Combined Cohorts). 


Table 5 below shows the calculations of the minimum detectable effect size at different points in the 
evaluation (as specified in the protocol): after randomisation, and with the final number of schools that 
were included in the ITT analyses. The large number of students enabled detection of a lower effect 
size. 
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Figure 2: Participant flow diagram for intention-to-treat analysis 


Treatment 


Allocation 


(school n = 24; 
pupil n = 6,960) 


Approached 
(school n ~ 1,000) 


Did not agree to 
participate (school n ~ 


Not meeting inclusion 
criteria (school n = 5) 


Agreed to participate 
(school n ~ 52) 


Randomised 
(school n = 47) 


Waitlist control 


(school n = 23; 
pupil n = 7,030) 


Lost to 
follow up 


(1 school 
dropped 
out, pupil 
n = 303) 


Follow-up 


Not analysed 


(pupil n = 
1,330) 
Includes 
missing KS2 
scores 


Post-test data 
collected 


(school n = 23; 
pupil n = 
6,657) 


Analysed 


(school n = 23; 
pupil n = 
5,327) 
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Lost to 
follow up 


(1 school 
closed, pupil 
n= 302) 


Not analysed 


(pupil n = 
1,670) 
Includes 
missing KS2 
scores 


Post-test data 
collected 


(school n = 22; 
pupil n = 6,728) 


Analysed 


(school n = 22; 


pupil n = 
5,058) 
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Table 5: Minimum detectable effect size at different stages 


Correlation Minimum 


Nl EXeq aToke) 29) oD 8] oy Ey 

(n = intervention; Deeneen Power Alpha detectable 
= pre-test & effect size 

La eMexe) 11 1Ke))) 


post-test (MDES) 


40 schools 
(20; 20) 


Protocol 


47 schools 
(23; 24) 


Randomisation 


Analysis (KS2 45 schools/ 
and GCSE 9910pupils 


English scores) | (23/5, 327; 22/5, 
058) 


Pupil characteristics 


TEEP was implemented in a wide variety of secondary schools across England. The intervention and 
control schools were closely matched in these key characteristics. 


While most of the schools had a moderate number of FSM-eligible children, it ranged from 13.6% to 
60.3%. There was an even wider range of EAL pupils—from 0.09% to 83.6% (see Table 6 below). We 
conducted a comparison of schools at baseline to see if there were significant differences between 
them on these characteristics. None of the differences between control and treatment schools at 
baseline were significant. We note that 32 of the 45 schools at post-test were academies. Some of them 
(four or five) became academies during the evaluation. We did not analyse the differences between 
academies and local authority controlled or religious schools. 


We were unable to collect Ofsted ratings for the schools as a number of schools changed status during 
the trial. 


Table 6: Baseline comparison 
WEVar le) (= aitcyaVtadtolame cele] e) (Orel alice) me] cole] o) 


STed aToye) Fa (-\V/-1| n/N (missing) Percentage AN Ci Lal )) Percentage | 


School average % of 5 
GCSE (A*-C) scores over 19/45 (3) 39.7% 20/45 (3) 38.3% 
three years 


School-level (continuous) ram Crales-iale)) Mean n (missing) Mean 


Number of Y11 pupils | 6,824(1,497) |  325.~—| 6,031. (973) a 


boYod aVoXo) Sy (=n(=1 im (or-\ccle ce) a (er-1) MAN (LIL ESILaTe Percentage n/N (missing) Percentage 


Eligible for FSM 19/45 (3) 28.75% 20/45 (3) 30.8% 


School-level (continuous) n/N (missing) Percentage n/N (missing) Percentage 


Percentage of EAL 
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Outcomes and analysis 


The analyses of the GCSE scores in English and maths (as described in Analysis section above), 
controlling for individual students’ KS2 attainment, found no positive impact for the TEEP programme 
(see Table 7 below). As the following tables show, neither did it have an impact for students eligible for 
free school meals, for either gender, or for students of different ability levels (See table 8 to 11 below). 
In some analyses the effect sizes show a small negative effect in favour of the control group, but these 
differences were not statistically significant. 


There was a high level of missing data from the analyses relating to, for example, students who dropped 
out of school or did not take the GCSE examinations. However, there was also a high number of missing 
KS2 scores in the sample, meaning that their data could not be included in the primary analysis. One 
possible explanation for the high level of missing KS2 data is that in 2010 approximately 60,000 children 
in England did not take the KS2 tests because schools did not administer them in protest to over-testing 
of children and the use of the KS2 scores to evaluate schools. 


When we conducted an analysis to determine whether there were significant differences between 
students in the treatment and control conditions for whom there were KS2 scores, there were 
differences on the English GCSE scores but not on the maths scores (see Appendix K). Students in 
control schools with KS2 scores had higher GCSE scores in English than those with missing KS2 
scores, while in treatment schools there was no correlation between the presence (or otherwise) of KS2 
scores and GCSE attainment. 


Table 7: Primary analysis 
Raw means Effect size 


Intervention group (ofol nl dae) me] cole] e) 


= Roan < Roan nin model Hedges p- 


NVEVUL =) 


Outcome (intervention; fe] 
(oxo) ay ine))) (95% Cl) 


(hal es-11 016) mm (ckok08 Ol) Mm (Lal eSS-1L 11 ) i (202008 © )] 


GCSE 6,657 6,728 10,385 
English (1,330) (1,670) (5,327; 5,058) 


GCSE 6,657 6,728 10,449 
Maths (1,262) (1,674) (5,395; 5,054) 


Table 6 above states the percentage of FSM students in the secondary schools at the beginning of 
the study. Table 8 below refers to the individual students ever being on FSM in their school life. 
Tables 8—10 below show the results for FSM children and for males and females, indicating small 
negative effects for those subgroups. 


Education Endowment Foundation 23 


The Teacher Effectiveness Enhancement Programme 


Table 8: Free School Meal analyses 
Raw means Effect size 


Intervention group Control group 


Mean (95% Mean ee Hedges g 


Cl) nD (95% Cl) ane (95% Cl) 


GCSE | 3,306 5,173 
English | (602) (2,704; 2,469) 


GCSE 3,306 5,212 
Maths (555) (2,751; 2,461) 


Outcome n 
’ 


Table 9: Analyses for males 
Raw means Effect size 


Taic=yaY(claitcelame |aelele) (orol alice) me] cele) e) 
Mean (95% Mean minmegs) Hedges g 


fe) A (95% Cl) pea (95% Cl) 


Outcome 


GCSE | 3,407 5,314 
English | (704) (2,703; 2,611) 


GCSE 3,407 
Maths (663) 


5354 
(2,744; 2,610) 


Table 10: Analyses for females 
Raw means Effect size 


Intervention group Control group 


Mean (95% Mean pinned: Hedges g 


Outcome fe} n (95% Cl) (intervention; (95% Cl) 


exeyayice))) 


n 
GCSE | 3,250 5,071 
English | (626) (2,624; 2,447) 
GCSE _ 3,250 5,095 
Maths | (599) (2,651; 2,444) 


Table 11 below includes only students with KS2 scores because it used those scores to calculate the 
ability groups, so there are no missing values. It shows small negative effects for all but the medium- 
ability students on GCSE English tests. 
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Table 11: Ability level analysis 
Raw means Effect size 


Intervention 


group 
Ability nin model Hedges 


Mean n Mean - ae 
Level Outcome n (95% Cl) (95% Cl) (intervention; 
refed alice) )) 


GCSE 3,444 
English | 1/826 tere (1,826; 1,618) 
GCSE 3,444 


GCSE 3,344 
_ English | 1/57 Les (1,757; 1,587) 


Control group 


GCSE 3,344 
GCSE 3,533 
English | 1*/46 ais (1,716; 1,817) 
GCSE 3,533 
1,716 1,817 Peer 


On-treatment analyses indicated essentially no difference between the 42 treatment and control schools 
that SSAT finally trained and that implemented the programme (see Table 12 below). 


Table 12: On-treatment results 
Raw means Effect size 


Intervention group | Control group 


n in model Hedges 
(intervention; fe] 
reroll ice) )) 


Mean n Mean 


n 
Outcome (missing) (95% Cl) (missing) (95% Cl) 


GCSE 6,466 6,256 9,865 
English | (1,750) (1,107) (4,716; 5,149) 


GCSE 6,466 6,256 9,976 
Maths (1,707) (1,039) (4,759; 5,217) 
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Cost 


Staff teaching time is not included in the cost calculation because the intervention replaces regular 
classroom teaching. Teachers in each intervention school spent three days attending training and the 
TEEP mentors in each school also participated in two days of additional training. Staff coverage is not 
calculated because the whole school staff is trained for two days during days set aside for CPD, when 
the school would be closed. A TEEP lead is required in each school. The cost of covering this person’s 
time would vary from school to school and would also depend on the professional standing of the 
person—whether subject lead, department head, or senior manager. It likely took extra time to prepare 
TEEP lessons but as we do not have this information it is not included in the costs. 


Schools need computer access for students, but we assumed that most schools had such access, so 
considered this a prerequisite cost. 


SSAT charges are based on the number of teachers and students in a school. For a school with up to 
50 staff members and 1,000 students, SSAT would charge about £13,000. In subsequent years SSAT 
provide ongoing support at no additional charge and provide online support. They have an established 
TEEP ambassador network, offer training, and champion schools that further support TEEP schools. 
We estimate the cost per school to be £13,000 which, if spread over three years, amounts to £4,333.00 
per year and equates to a very low cost of £4.33 per student (See Table 13 below). These are 
approximate figures as they depend on a number of variables, mostly the number of attendees at the 
training sessions. 


Table 13: Cumulative and average per-school and per-pupil costs for TEEP over three years 
Year 1 Year 2 Year 3 Average over 3 
Cumulative years 


pol £13,000 £13,000 £4,333 
school 


Perpupl| #1300 | 0 | 0 | _#48.00 A 
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Process evaluation 


Members of the research team were able to examine the implementation of the TEEP programme 
through the scrutiny of an extensive amount of documentation provided by SSAT (such as handouts 
from training sessions) and the schools (such as handouts used in class lessons that were observed). 
This documentation included notes from participation in TEEP training by two research team members; 
interviews with TEEP trainers, TEEP school leads, school senior managers and teachers; detailed 
lesson observations; and online surveys with teachers and students, all described above. 


The process evaluation was conducted in randomly selected intervention schools so as to be able to 
study TEEP in some depth and over a reasonable period of time. Researchers visited these schools 
before they commenced involvement in TEEP (pre-training visits) and attended two training sessions. 
They then returned during implementation of TEEP to be able to follow progress over time. 


Online teacher surveys were completed by 422 teachers from 15 intervention schools. Online student 
surveys were completed by 2,440 students from 13 intervention schools. The ratings by teachers and 
students cannot be related directly to the attainment outcomes because in some schools the surveys 
were completed across all year levels (Years 7 to 11) whereas in others only by students in the year 
level (Year 11) being assessed. 


As detailed below, the process evaluation does not shed too much light on why the impact results were 
not more positive. The buy-in from senior leadership and teachers was good—headteachers and 
department heads were enthusiastic and thought that TEEP would directly impact on teaching and 
improve practice. The training was also well-received by the teachers and of a high quality as evaluated 
by the researchers in the randomly assigned schools that were observed. The implementation was 
rated as relatively high in the limited amount of data that was collected from the lesson observations of 
this small number of schools. 


Pre-training school visits 


During the pre-training visits to the TEEP schools (at the stage when schools had signed up for TEEP 
but had not yet started their involvement) the researchers sought to find out about the school’s priorities, 
its current and recent improvement initiatives or programmes, and its reasons for electing to take part 
in the TEEP programme. 


All of the headteachers interviewed commented along the lines that their reasons for joining TEEP were 
that it had ‘a focus on teaching and learning’ and/or that there was a need to improve teaching in their 
school (in other words, to increase the number of teachers rated ‘good’ or ‘outstanding’). Some others 
added more practical reasons, such as ‘we were below the floor targets for proportion achieving five 
good GCSE grades’. 


Another common finding at the pre-training stage was that the school managers felt that the time was 
right for TEEP, or that it fit well with what the school was planning to do. A typical comment was, ‘TEEP 
is the next stage, it will take us to the next level’. 


TEEP training 


Observations of training sessions 


The training sessions observed were delivered skilfully. Where staff raised concerns during the training 
sessions—such as ‘Can TEEP be applied to all subjects?’, ‘Why do TAs need to be involved?’, ‘ls TEEP 
lesson preparation more time-consuming?’—the issues were addressed effectively. The trainers were 
complementary about the schools’ commitment and the teachers’ enthusiasm for TEEP. 
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All schools and teacher participants were provided with common documentation by SSAT emphasising 
the fundamental teacher and learner behaviours which form the base of the model, the cultural changes, 
the core elements, and the TEEP Cycle (see description of intervention above). The training sessions 
attended by the researchers were very similar and both assessed to be of very good quality. There was 
consistency in the delivery of the training even though different trainers were observed. The quality of 
the training observed suggested that the training had been carefully devised and quality assured in 
order to ensure consistency across trainers. 


TEEP was frequently described by the trainers and participants as ‘a journey’, which was reflected in 
the SSAT documentation. So, unlike some CPD interventions, this was not a one-off; rather, the 
programme had long-term goals and was seen as part of a process of continuous improvement. This 
means that although implementation had a clear starting activity, once the programme was underway 
it was expected to spread across the whole school and to have an increasing impact over time. Through 
ongoing CPD, mostly conducted by the TEEP lead in each school, teachers would become more 
proficient in integrating TEEP elements into their teaching. 


Teachers’ perceptions of training 


The teacher surveys indicated that 79% of the respondents perceived the training as useful. The training 
sessions were well received by staff in both schools where training was observed. The staff could see 
the benefits of having a common language to address teaching and learning needs, and enjoyed 
discussing the TEEP core elements and the stages of the TEEP cycle. While there was much 
information to take in over the training, a large majority of staff indicated that they would be seeking to 
implement at least some of the principles of TEEP in forthcoming lessons. IEE researchers also 
attended the third day of the training for the TEEP leads in the two schools to see how the two schools 
had begun to take TEEP forward and whether or not there were early signs that TEEP was becoming 
embedded in the schools. The TEEP leads were very enthusiastic and positive about the training. They 
appreciated that teachers were not expected to immediately change all of their teaching to follow the 
TEEP model but that it was intended to be gradual process, integrating elements of TEEP into their 
lessons over time. 


Implementation fidelity 


Because TEEP is a whole-school model it was generally implemented across the year levels. There 
was anecdotal evidence from the school visits and interviews that a good level of implementation was 
maintained, however, there were no objective implementation fidelity assessments conducted by the 
TEEP leaders or trainers or the evaluation team. After the whole school training was completed, which 
included all the teachers in the school, the teachers were encouraged to build the five core elements 
into their lesson plans. The lack of implementation fidelity rating makes it very difficult to establish 
whether the programme itself or poor implementation is responsible for the results. It was assumed that 
TEEP would be adopted gradually. Teachers were also encouraged to share ideas with each other 
about how they had incorporated TEEP into their lessons. Of course, how teachers did this was subject 
to a good deal of variation and this was expected by the TEEP trainers. There was variation among 
different subjects, different year levels, and different levels of enthusiasm for some aspects of TEEP 
over others. Because observations were conducted across all these variables it is impossible to 
determine any correlation between the variables and implementation fidelity or students’ attainment. 
Trainers stated, for example, that it was not possible build all of the five TEEP elements into one lesson 
plan. 
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Outcomes 
Lesson observations 


Over time, 16 lessons were observed in eight visits to randomly-selected implementation schools. Some 
schools were observed more than once but different teachers were observed in each visit. Because 
teachers varied in which subjects they taught, which year levels, and which ability groups, it was 
impossible to perceive trends in the level of TEEP implementation fidelity over time. In all of the classes 
observed elements of TEEP were evident, though to varying degrees. Due to the range of subjects, 
year groups, and abilities observed, it is difficult to draw out how effectively TEEP was implemented 
across this range. Also, the scope of TEEP meant that a lot of different lessons were observed. For 
example, one teacher was seen to be employing co-operative learning in an observed lesson and 
embedding elements of assessment for learning. In another observed lesson, the teacher was using 
ICT throughout the lesson. 


Classroom climate and classroom management were rated very highly in most of the observations. The 
level of interactivity varied considerably. In some classes students were actively engaged in discussion 
and in others they were doing mostly individual seatwork. From our lesson observations, researchers 
had the impression that there were variations by subject and teacher experience. 


Students appeared to be very engaged in the lessons, making decisions and asking questions. Students 
checking their progress and displaying strategies for learning were less apparent but it may be that 
those behaviours are just more difficult to observe. 


These observations must be interpreted with caution. The researchers only observed 16 lessons in 8 
different schools and in a number of different year levels and subjects. They may or may not have been 
representative of typical lessons in the TEEP schools. There is no evidence for this, but one might 
expect teachers to put extra planning and care into lessons that they knew were going to be observed. 


Teacher surveys, interviews, and focus groups. 


Overall, teacher perceptions of the TEEP programme were very positive from the surveys, the 
interviews, and the focus groups. They reported that TEEP improved their confidence (58% either 
agreeing or strongly agreeing). Between 55% and 71% of teachers either agreed or strongly agreed 
that four of the five components (Assessment for Learning, Collaborative Learning, Accelerated 
Learning, and Thinking for Learning) were enhanced by implementing TEEP. The only component that 
the teachers reported as not being as effective was the experience of TEEP in ensuring that they made 
more effective use of ICT in their teaching. Only 37% agreed or strongly agreed. 


The majority of teachers reported being on board with TEEP and found it reinvigorating. It helped with 
lesson planning and the TEEP resources were welcome. They felt it engaged the students, made them 
more active learners, and gave them increased responsibility for their learning. Teachers reported that 
fewer students were being referred for misbehaviour. 


The different elements were not that new; it was putting them all together that was novel. They started 
out gradually introducing the elements until they became confident and then added more into their 
lessons. Teachers reported that it helped that they talked less and were more effective in questioning. 
They reported that it helped having everyone in the school using the same language. It gave the 
teachers more confidence. 


A few teachers reported finding TEEP a bit overwhelming at the beginning, though the evaluation did 
not follow these teachers to see if they found implementation easier over time. They reported that it was 
hard to find time to fit in all the interactive aspects of TEEP approaches (such as feedback to students 
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or collaborative learning) into one lesson period, which is not the intention of the TEEP approach 
anyway. A few said it was hard to not to fall back into their old ways. Some schools did not have the 
equipment to do all the ICT teaching that TEEP called for. Some teachers did not have the expertise, 
nor the confidence, to engage with the ICT component. A few teachers were rather cynical and suffered 
from ‘strategy overload’. 


Headteacher/Deputy Headteacher and Head of Department interviews 


The interviews with the school leaders showed that the TEEP elements aligned closely with the schools’ 
priorities. The school leadership teams reported wanting improve their students’ attainment and their 
Ofsted ratings. Some reported already using coaching to improve teaching. They believed that TEEP 
would help them have a stronger focus on assessment for learning and to have a more systematic 
model of lesson planning to create more interactive lessons. It was reported that this happened more 
in English and in maths, though somewhat less in science. However, there was no comparison made 
between subjects as the numbers were too low to be meaningful. 


The interviewees said that TEEP had enhanced the co-operative learning that was taking place, which 
improved the students’ engagement. The only area where a few school leaders reported that they 
weren't making as much progress embedding TEEP was in ICT. 


Student perceptions 
Student surveys 


In the student surveys, the students were asked to reflect on how TEEP had impacted on their 
effectiveness as learners. Thirteen out of 22 TEEP schools had their students complete the surveys. 
Schools varied in year levels that they included in the evaluation—from including all year levels to only 
including students in one. This makes a comparison among the schools impossible. 


By and large, students believed that participation in TEEP improved their effectiveness as learners, with 
58% ‘strongly agreeing’ or ‘agreeing’ with the statement ‘All round, | am a more effective learner than 
before’, compared with 12% who ‘disagreed’ or ‘strongly disagreed’. 


On other ratings related to the impact of TEEP on confidence, co-operation, working things out, whether 
lessons were ‘interesting’, and so on, many more students expressed positive opinions (approximately 
44%) compared to negative opinions (approximately 15%). 


Control group activity 


Two schools visits and seven telephone interviews with control school headteachers, deputy heads, 
and department heads revealed that while control schools were implementing various strategies to 
improve their attainment, their interventions did not resemble TEEP in terms of its whole-school, 
comprehensive approach. They often reported that increasing feedback to students (for example, using 
assessment for learning) or improving interaction in the classroom (for example, by using collaborative 
learning) were teaching and learning goals for the school. Because ICT is increasingly being used in 
schools, and meta-cognition and assessment for learning are currently popular strategies, some of the 
schools reported implementing them in some of their classes; however, none of the schools reported 
using TEEP. In the few control-school lessons that were observed, a couple of the strategies were 
implemented but there was no consistent use of the combination of strategies that are representative 
of the TEEP approach. 


The lessons observed in the intervention schools, therefore, appeared to contain more of the elements 
of TEEP than those in the control schools. However, the limited number of observations in both 
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intervention and control schools means that it is impossible to conclude without doubt that intervention 
schools were definitively more ‘TEEP-like’ than the control schools. 
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Conclusion 


1. In the low-performing schools selected for this trial, there was no evidence of an impact on 
pupils’ GCSE English and maths attainment in schools selected to receive TEEP training 
compared to other schools. 


2. Both teachers and students were enthusiastic about the programme and believed that it 
improved students’ learning. 


3. The evaluation was initially designed to also assess implementation quality and the impact 
on Year 9 attainment and attitudes. However, the relevant measures for these key evaluation 
elements were not collected due to circumstances beyond the control of the project teams. 
This means the evaluation cannot assess whether TEEP made a difference to Year 9 pupils, 
as originally intended. 


4. Any future studies could systematically investigate the implementation of TEEP using 
implementation measures that relate to specific components of the programme, as well as 
investigating changes in school culture and teacher behaviour. 


Interpretation 


Given the strength of evidence on the Teaching and Learning Toolkit behind the components of TEEP, 
such as collaborative learning and assessment for learning, it is surprising not to find a positive impact 
on the GCSE scores of students in the TEEP schools. Also, based on the findings of previous studies 
of TEEP that showed mixed but mostly positive qualitative findings, one would expect there to have 
been a positive impact on achievement. 


TEEP is a complex intervention with a lot of moving parts. Because it involves changing so many 
elements of teaching (for example, assessment for learning, co-operative learning, and ICT) it may be 
that it takes a longer time to have an impact on GCSE scores. The TEEP model calls for a gradual 
inclusion of elements into teachers’ lessons: it might be too much to expect to be able to assess impact 
after only one and a half or two years’ implementation. The implications for development might be to 
introduce TEEP gradually over a longer time. However, one previous evaluation found noticeable 
changes in teaching behaviour, aligned with the TEEP approach. 


Because this was a well powered trial, but with considerable attrition, the findings for the primary 
outcomes are of moderate security. The trial involved schools in many regions of England, with varying 
demographics, so findings are likely quite generalizable to low performing schools in England. 


Limitations 


The initial plan had been to assess the impact of TEEP on the CEM INSIGHT achievement test as 
secondary outcome measures in addition to analysing the GCSE English and maths scores. However, 
there were unavoidable problems with the external testing and communications around these. In June 
2014 only five schools from a possible 18 completed tests, and in June 2015 only 14 schools out of a 
possible 27 completed some parts of the tests. This represented considerable missing data. At the July 
2015 meeting with EEF it was agreed that, as was originally planned, the analysis would be of GCSE 
data as the main outcome measure and to not analyse the post-test INSIGHT data. 


It may be that the CEM tests might have been more sensitive to the whole-school impact of the TEEP 
programme, particularly in the early year levels, however TEEP is a whole-school intervention and was 
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being implemented at all year levels in most of the treatment schools. Having outcome measures that 
assess the impact of TEEP on specific outcomes that TEEP targets might be a good next step. 


The process evaluation indicated that the ICT take-up was not as thoroughly embedded as some of the 
other elements. However, we do not have clear indication how well the elements were implemented 
across all of the schools as only eight schools were visited and SSAT says that not all elements would 
be implemented in each lesson. We had planned to calculate the correlation between level of 
implementation fidelity ina school and the student GCSE attainment, however there were no consistent 
objective assessments of implementation fidelity so it was not possible to conduct that analysis. The 
interaction effect on the GCSE English scores for students with missing and available KS2 data and 
being in treatment and control schools makes interpreting the results more tentative. Students in control 
schools had higher GCSE scores in English, but not maths, if their KS2 scores were available compared 
to those students whose KS2 scores were missing, while the availability of KS2 scores had no impact 
on the GCSE scores of students in treatment schools. 


Regretfully, the loss of the PI on this project definitely had a negative impact on its conduct, particularly 
on the process evaluation. She had established good relationships with the participating schools and 
that was difficult to replicate in the time remaining for the project. Other staff changes at the IEE delayed 
the submission of the final report. 


Future research and publications 


Conducting a large randomised controlled trial with a complex programme such as TEEP is a challenge 
for all of the participants. It is an important step forward, but only focused on one year group. Conducting 
an evaluation of TEEP on all of the students over a longer period of time might be an appropriate next 
step. For example, one could compare the GCSE scores of the students who were exposed to four 
years of TEEP to those who were exposed to only two years to gain some information about the length 
of time it takes to embed TEEP. However, this would not be a randomised evaluation, without a pure 
control group. 


In order to achieve sufficient power for the evaluation, we only evaluated TEEP in secondary schools. 
It might be that it is more effective in primary schools. An evaluation of TEEP implemented in primary 
schools might be worth considering. 


Finally, future evaluations should seek to unpick the various factors of implementation that might make 
this intervention successful (or, conversely, unsuccessful). Future evaluations of TEEP should seek to 
collect both implementation data (relating to the fidelity and quality of the implementation and the 
dosage) alongside detailed measures of teacher and pupil behaviour. 
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Appendix A: EEF cost rating 


Cost ratings are based on the approximate cost per pupil per year of implementing the intervention over 
three years. More information about the EEF’s approach to cost evaluation can be found here Cost 
ratings are awarded as follows: 


Cost rating Description 


ELEELE Very low: less than £80 per pupil per year. 
ELELEE Low: up to about £200 per pupil per year. 


ELEELE Moderate: up to about £700 per pupil per year. 
ELEEEEL High: up to £1,200 per pupil per year. 
ELEELEE Very high: over £1,200 per pupil per year. 
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Appendix B: Security classification of trial findings 


Ra ¢ aie wiela er: ¢ : = Adjust 
Design Power | Attrition* 
5) Fears = =6—hf ietté‘iL:*‘“‘iUC 
0-10% 

4@ |Fairandclearquas- | | | | | [o™ 
experimental design for 
comparison (e.g. RDD) with Adjustment 
appropriate analysis, or MDES< | 44-20% for 
experimental design with 0.3 Balance 
minor concerns about 
validity [] 


Well-matched comparison 
(using propensity score 


matching, or similar) or MDES < 
experimental design with 0.4 
moderate concerns about 
validity ; 
ed Adjustment 
2@ | Weakly matched comparison for threats 
or experimental design with MDES < 31-40% to internal 
major flaws 0.5 validity 
UD Ea ee To 
Comparison group with poor 
or no matching (E.g. MDES < 51-50% 
volunteer versus others) 0.6 
Os 
No comparator ee e <50% 


e Initial padlock score: lowest of the three ratings for design, power and attrition = 3 padlocks 
e Reason for adjustment for balance (if made): n/a 

e Reason for adjustment for threats to validity (if made): n/a 

e Final padlock score: initial score adjusted for balance and internal validity = 3 padlocks 


e = The trial was a school level randomised effectiveness trial. 

e The sample size was able to detect a MDES of less than 0.2 at analysis 

e §=6Asaresult of missing data and attrition there was relatively high data missing between randomisation 
and analysed - 25.7% at the pupil level (13,990 pupils randomised, 10,385 analysed). 

e The samples seemed relatively well balanced at baseline (though they do not present analysis of KS2 
results). 

e = Threats to validity: no clear indication of counterfactual, but TEEP suitably complex enough for us to 
assume that control schools were not doing TEEP 
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Appendix C. Head Teacher Agreement 


Evaluation of Teacher Effectiveness Enhancement Programe (TEEP), 2012-2015 
Dear Head Teacher, 


Researchers at the Institute for Effective Education (IEE) will conduct an evaluation on behalf of the 
Education Endowment Fund and The Schools Network to assess whether TEEP improves children’s 
learning and attainment. The evaluation will start in the Autumn Term in the school year 2012-2013 
and will finish at the end of the Summer Term, 2015. Two cohorts of schools will take part. The first 
cohort will participate in the evaluation from Autumn Term 2012 to Summer Term 2014. The second 
cohort will participate in the evaluation from Spring Term 2013 to Summer Term 2015. 


The researchers at the IEE will randomly assign schools to the experimental group to use the TEEP 
programme for two years or to the control group to continue with their regular teaching and learning 
programmes and strategies for two years and to implement TEEP thereafter. 


Training for the experimental group of schools who will be implementing TEEP will be provided in all 
five strands of TEEP prior to the implementation of the programme. 


For the first cohort, in the Autumn term 2012/2013, prior to implementation, Year 7 pupils in both the 
experimental and control schools will complete a test to provide a base-line measure. In the Summer 
term of school year 2013-2014, the Year 9 pupils in both the experimental and control schools in the 
first cohort will complete a post-test to provide comparison data. The process will be repeated for 
schools in the second cohort. 


During the evaluation, head teachers, teachers and students will be surveyed on-line three times 
about their perceptions of the efficacy of TEEP. Lesson observations in a random selection of lessons 
and focus group interviews with samples of teachers will be conducted on three occasions over the 
implementation period. 


The study will provide valuable information about the effectiveness of the programme for improving 
learning and raising pupil attainment. 


If you have any questions or want to discuss the project, you can contact me at 
mary.sheard@york.ac.uk or call me at 01904 328159. Please complete and return the form below if 
you agree to take part in the evaluation. 


We look forward to hearing from you. 


Sincerely, 


M tangethetaA- 


Mary Sheard 
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Terms and Conditions of Participation in the Evaluation 


Please complete and return the form to the address below if you agree to take part in the 
evaluation of TEEP and accept terms and conditions of the evaluation. 


| agree for my school to take part in the TEEP evaluation and | accept the terms and conditions. 


Name of HeadTeacher: 


Signature of Head Teacher: 


School address: 


School email: 


School FAX: 


School telephone 
number: 


Main contact for the 
evaluation: 


Contact details: 


Number of students on 
roll: 
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Local Authority: 


Please return by email to mary.sheard@york.ac.uk or by FAX (01904 328156), or post to Mary 
Sheard at the address below. 
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Appendix D. Parent Information Sheet and Opt-out Form 


Institute for effective education, the University of York 


Teacher Effectiveness Enhancement Programme (TEEP) 


Information Sheet for Parents/Guardians 


Dear Parent/Guardian, 


We would like to request your permission for your child to take part in an educational evaluation 
study. The following information explains why the research is being done and what it would involve for 
your child. 


What is the Institute for Effective Education? 


The Institute for Effective Education (IEE) is part of the University of York. It aims to find out what 
works in teaching and learning and why, and then use the evidence to improve education. 


What is the purpose of this study? 


This study is being done to see if using a professional development programme for teachers (TEEP) 
helps students to achieve better results. 


Why is my child’s class participating? 


We will conduct this study with Year 7 classes in 40 secondary schools in England. The headteacher 
of your child’s school agreed to participate in this study. 


Does my child have to take part? 
At the beginning and towards the end of the evaluation, 


Your child’s test scores will be confidential and will not count towards your child’s assessment scores 
or levels in school. You may choose not to permit your child’s test scores to be used in the evaluation. 


If so, please complete and sign the attached opt-out form by 3:00 pm on Monday (insert date here) 
2012. A pupil's right to withdraw from the testing will be respected. 


What will my child’s participation be? 
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This Summer Term, and again in the Summer Term 2015, the current Year 7 students will do an 
online test that includes items assessing English, maths and general ability. We will use the test 
results to see if the teachers’ professional development programme has improved student learning. 


What should | tell my child about the study? 


It would be helpful if you could tell your child that the study is trying to find out whether a particular 
professional development programme for teachers helps students to learn and achieve better. 


What are the disadvantages and risks of taking part? 


There are no known disadvantages or risks in participating in this study. The test data will be 
securely stored and managed, scored electronically at the Centre for Evaluation and Monitoring 
(CEM), University of Durham, and will not be shared with anyone. Teachers will continue to teach to 
the usual lesson objectives throughout the evaluation period. 


What are the possible benefits of taking part? 


By participating in this study, your child will experience an up-to-date online approach to assessment. 
The information gained from this study may influence how your child and others will be taught more 
effectively in the future. 


What happens when the research stops? 


IEE researchers will analyse the test scores to determine the overall effectiveness of the TEEP 
professional development programme for teachers. Scores for individual pupils and classes will be 
kept confidential. 


When the research is over, the school will receive a report that will show if using TEEP makes a 
difference to pupils’ achievement. The school can then decide whether and how best to use the TEEP 
programme in the future. 


Will my child’s information be kept confidential? 


Yes. Students’ names will be replaced with code numbers. No individual pupil’s data will appear in 
any report. The research team will not have access to pupil or parent names. 


What if there is a problem? 


If you have a concern or question about your child’s participation in this study, please contact Bette 
Chambers (e-mail: bette.chambers@york.ac.uk) Tel: 01904 328153 or Mary Sheard (e-mail: 
mary.sheard@york.ac.uk) Tel: 01904 328159. 
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THE UNIVERSITY Of York 


Institute for Effective Education 


Parent/Guardian opt-out form 


If you do not permit your child’s test scores to be used in the study, please complete this form and 
return it to your child’s teacher by 3:00 pm on Monday (insert date here), 2012. 


| do not wish my child’s test scores be used in the study. 


Pupil’s name? As. secs eee eee We ea Sea 


(Please print clearly) 


Teacher's: Name? zi fiscedoeei ie deosls eee Muneades sediedos even beakeadoet 

MC MOO ei ee cetacean Oe 8 al ae eee nol oh Meh neon ad sete, tueaeet gate Dees, geek tee Boas 
Parent’s/Guardian’S NAME? ...........c cece ceccccc ee ccceceecseseeeeeeeesesuueeeeuaaeeeenaneas 

(Please print clearly) 


Parent’s/Guardian’s Signature: ..............cccccnee eect eee eeetneeeeeeetnaeeeeenenaes 


Date ei igiei tied cccetis fava pointed iva eae Mactanddas Pe seek ak aad aed RaNE vA Ga oad aka HN Daas a ea ead eA ey 
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Observer Date 
School Subject 
Year Group Teacher 
Pupil Grouping No. of 
Students 


Lesson Focus 


Lesson Plan 


As of 
Underpinning 
Elements 


0-5 
mins 


6-10 


11- 
15 


16- 
20 


21- 
25 


26- 
30 


31- 
35 


36- 41- 
40 45 


46- 
50 


51- 
55 


56- 
60 


Assessment for 
learning 


Collaborative 
learning 


Accelerated 
learning 


Thinking for 
learning 


Effective use of ICT 


As of Planning 
Cycle 


0-5 
mins 


6-10 


11- 
15 


16- 
20 


21- 
25 


26- 
30 


31- 
35 


36- 41- 
40 45 


46- 
50 


51- 
55 


56- 
60 
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Prepare for learning 


Agree learning 
outcomes 


Present new 
information 


Construct meaning 


Apply and 
demonstrate 
understanding 


Review 

Effective teacher behaviours Observations 
Variety of teaching styles 

*Plan lessons that are inclusive ba 

*New information presented in a variety of ways - 

*Lesson moves at brisk pace 

*Students have opportunity to work and learn with each other * 

*Students are given time to explore new concepts before being assessed * 
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Interactive teaching 


*Questions used by teachers and students as tools for learning * 
* ‘Wait time’ is observed; students given time to formulate their responses 3 
*Regular reviews to consolidate what has been learned . 
*Discussion — students share their ideas and have the chance to listen and 2 


learn from each other 


Classroom management 


*Clear and consistent behaviour strategies . 
*Effective classroom organisation * 
*Lessons are well planned and structured % 
*Teacher gains and holds students’ attention * 
*Routines and structures are establishes to support the learning process * 


Classroom climate 


*High expectations ia 

*Strong relationships % 

*Enthusiasm for teaching and learning * 

*Safe environment, physically and emotionally * 

*Student engagement-students actively participate e 

Effective learner behaviours Saliency Proportion of students 

Makes decisions 0 A 2 3 0 % 7 % All 
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Seeks assistance 0 1 2 3 0 % 7 % All 
Checks personal progress 0 1 2 3 0 Y% Ye % ~~ ©All 
Asks questions 0 1 2 3 0 Y% 7) % ©All 
Makes links to previous learning 0 a 2 3 0 Y% Ye % ~~ All 
Has strategies for learning 0 1 2 3 0 Y% 7) % ~~ ©All 
Reflects on the work 0 a 2 3 0 % i 7 All 
Considers how they have learned 0 1 2 3 0 Y% 7) % All 
Additional Observer Comments Teacher Quotes 
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1. Deployment of Teaching Assistant 
2. 


3. 


Student Quotes 
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Appendix F. Head Teacher Baseline Interview Questions 


TEEP Evaluation: Summer Term School Visits 


Interview with Head teacher/Deputy Head teacher 


1. 


Using the school’s recruitment profile information, a short introductory discussion will focus on 
the reasons why the school elected to take part in the TEEP project. Following this, the 
interview will address the questions below. 


What are the school’s current priorities for improvement? 


What are the schools’ current policies and practices in the following areas and what changes 
are intended as a result of TEEP? 


Assessment for learning; 
Thinking for learning; 
Accelerated learning; 
Collaborative learning; 


Use of ICT. 


What will be the mechanisms for change in teaching and learning in the areas identified in 


Q3? 


Will change in teaching and learning be evaluated at school and department level? If so, how 
will change be recorded/documented? 


Will you/SMT identify the impact of TEEP on student engagement, progress and attainment? 
If so, how? 


What evidence will you/SMT seek to demonstrate the effectiveness of TEEP on teaching and 
learning? 
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Appendix G. Baseline Focus Group Questions 


Focus group questions with HoDs and/or teachers: e.g. English, maths, science 


10. What are your departments’ current priorities for improving teaching and learning? 


11. What if any are your expectations of the TEEP project for enhancing teaching and learning in 
your department/subject area? 


12. What are your departments’ current policies and practices in the following areas, and what if 
any changes in teaching and learning are you intending as a result of TEEP? 


e Assessment for learning; 
e = Thinking for learning; 

e Accelerated learning; 

e Collaborative learning; 


e Use of ICT. 


13. What will be the mechanisms for change in teaching and learning at department level? How 
will change be recorded/documented? 


14. How do you think TEEP will be directly benefit students’ learning and attainment in your 
subject areas? 


15. Do you envisage any potential challenges in implementing TEEP in your departments? If so, 
what might they be and how will you address them? 
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Appendix H. TEEP Teacher Survey Questions 


Your school’s name 


Your job title 


Gender: please tick one; male female 


Age group: Under 25 years 25-34 35-44 45-54 Over 54 


Main subject taught: 


(Any) Other subjects taught: 


Month & Year of first TEEP training session: Month Year 


How, if at all, has your teaching changed from the period before you had any TEEP training to the 
period after you completed TEEP training? (Please tick one box in each row). 


Now Strongly | Agree Not Disagree Strongly 
agree sure disagree 


| talk at the class less than | used to 
My lessons include a bigger range of activities than before 


| use ICT/computers more effectively (for teaching and 
learning) than | used to 


Students now take a more active part in lessons than they 
did previously 


There is now more working together by students 


My lessons are more interesting than they used to be 


| feel better motivated to teach than | did before 


Students ask more questions than they did previously 


Students do more working out of things for themselves 
than they did before 


All round, | am a more effective teacher than before 


Please indicate the extent to which you agree with each of the following statements (tick one box 
in each row). 
The experience of the TEEP training programme has... Strongly | Agree | Not | Disagree | Strongly 


agree sure disagree 


.. improved my confidence as a teacher 


.. brought about more variety in my teaching styles 


.. helped me to plan lessons better 


.. led to a better ‘climate for learning’ in my classroom 


.. improved my classroom management skills 


.. improved my understanding of Assessment for Learning 


.. enabled me to facilitate more collaborative learning 


.. improved my understanding of ‘accelerated learning’ 


.. has helped me to better deliver ‘thinking for learning’ 


.. has ensured that | make more effective use of ICT 
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3. Please tell us about one lesson that you have delivered in the past few weeks that you feel was 
influenced by TEEP and was also an interesting lesson for the students. 


(a) What was the subject (e.g. science)? 
(b) What was the topic of the lesson? (e.g. learning about the human body) 


(c) What activities were used in the lesson? (e.g. teacher talking, discussion, video, worksheet...) 


(d) Why do you feel that this lesson was particularly interesting? 


(e) What did the students learn from this lesson? 


4. On ascale of 1 to 10, where 1 means ‘no support at all’ and 10 means excellent support, 
please rate the support you have received from your TEEP mentor. If you are a TEEP mentor 
please circle A, or if you do not have a TEEP mentor please circle B. 


1 2 3 4 5 6 7 8 9 10 A B 


To what extent do you feel that your school management are committed to the TEEP 
Programme? 1 means ‘not at all committed’ and 10 means ‘totally committed’. 
1 2 3 4 5 6 7 8 9 10 


5. Overall, on a scale of 1 to 10, where 1 means ‘not at all useful’ and 10 means ‘extremely 
useful’, how would you rate the TEEP training programme? 


1 2 3 4 5 6 7 8 9 10 
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6. To what extent do you feel that the TEEP programme has changed the way that you teach? 
Please rate this on the following scale, where 1 means ‘no change at all’ and 10 means ‘TEEP 
has completely transformed the way that | teach’: 


1 2 3 4 5 6 7 8 9 10 
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Appendix I. TEEP Student Survey Questions 


Your school’s name 


Your year group 


Gender: please tick one; male female 


1. We would like to know about how you learn in the three core subjects of Maths, English and 
Science, starting with Maths (please tick one box in each row) 

In Maths... Never Sometimes | Often | Always 
| check my work and correct mistakes 

| plan my work carefully before starting 

| can see links between the work and my personal life 

| have opportunities to work with other students 

| tell the teacher if | don’t understand something 

| can take risks and suggest answers that might be 
wrong 

The use of ICT/computers helps my learning 

The teacher makes the lesson interesting and varied 

| have the freedom to make my own decisions 

| can reflect (think about what | have done in the lesson) 


2. Please now answer these questions in relation to your English lessons (tick one box in each row) 
In English... Never Sometimes | Often | Always 
| check my work and correct mistakes 

| plan my work carefully before starting 

| can see links between the work and my personal life 

| have opportunities to work with other students 

| tell the teacher if | don’t understand something 

| can take risks and suggest answers that might be 
wrong 

The use of ICT/computers helps my learning 

The teacher makes the lesson interesting and varied 

| have the freedom to make my own decisions 

| can reflect (think about what | have done in the lesson) 


3. Please now answer these questions in relation to your Science lessons (tick one box in each row) 
In Maths... Never Sometimes | Often | Always 

| check my work and correct mistakes 

| plan my work carefully before starting 

| can see links between the work and my personal life 

| have opportunities to work with other students 

| tell the teacher if | don’t understand something 

| can take risks and suggest answers that might be 

wrong 

The use of ICT/computers helps my learning 
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The teacher makes the lesson interesting and varied 


| have the freedom to make my own decisions 


| can reflect (think about what | have done in the lesson) 


4. We’d now like you to think carefully about your learning in general (i.e. across all subjects): 
think about what your learning is like now compared to one year ago. (Please look at the 


following statements and tick one box in each row). 


Now... 


Strongly 
agree 


Agree 


Not 
sure 


Disagree 


Strongly 
disagree 


Our teachers talk at us less than they used to 


Lessons include a bigger range of activities than before 


We use ICT/computers more effectively than we used to 
| take a more active part in lessons than | did previously 


There is more working together by students 


Lessons are more interesting than they used to be 


| feel better motivated to learn than | did before 


| ask more questions than | did a year ago 


We do more working out of things than we did before 


All round, | am a more effective learner than before 


5. Please tell us about one interesting lesson that you have had in the past few weeks. 


(f) What was the subject (e.g. science)? 


(g) What was the topic of the lesson? (e.g. learning about the human body) 


(h) What activities were used in the lesson? (e.g. teacher talking, discussion, video, worksheet...) 


(i) Why was this lesson particularly interesting? 
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Appendix J. Comparison of the Cohorts 1 and 2 


Intervention Control 
Cohort Out- n mean mean nin model Effect Size 
fore} aat=) (miss) 
95% Cl 95% Cl | Cc 95% Cl 
1 GCSE 3,310 36.12 3,317 37.05 6,147 -0.03 0.58 
English 
(281) 35.70 | 36.54) (199) | 36.66 37.45 | 3,029 | 3,118 | -0.15 0.09 
GCSE 3,310 32.82 3,317 33.74 6,140 -0.04 0.50 
Maths 
(284) 32.34 | 33.29 (203) | 33.26 34.22 | 3,026 3,114 | -0.16 0.08 
2 GCSE 3,347 34.47 3,411 36.05 4,238 -0.06 0.55 
English 
(1,049) 34.00 34.94 | (1,471) | 35.61 36.49 | 2,298 1,940 | -0.26 | 0.14 
GCSE 3,347 34.00 3,411 34.92 4,309 -0.01 0.92 
Maths 
(978) 33.52 34.48 | (1,471) | 34.43 35.40 2,369 1,940  -0.13 | 0.15 
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The tables below show the mean and SE for the Intervention and Control treatments for those 
students where KS2 English is missing and where KS2 Maths is missing. The tables give similar 
results because a pupil who is missing one score is usually missing the other score (Both KS2 score 
available, N=10893; neither available, N=2985; English only available, N=71; Maths only available, 


N=133). 
KS2 English missing KS2 English available Interaction 
Intervention Control C-l Intervention Control C-| 
mean mean mean mean mean mean wate 
SE SE SE SE SE SE P 
GCSE 32.14 34.55 2.41 35.91 36.86 0.95 .007 
English 0.76 0.73 1.06 0.69 0.69 0.98 
GCSE 32.95 34.43 1.48 33.38 34.21 0.84 .290 
Maths 0.72 0.69 1.00 0.63 0.62 0.88 
KS2 Maths missing KS2 Maths available Interaction 
Intervention Control C-l Intervention Control C-| 
mean mean mean mean mean mean ealtia 
SE SE SE SE SE SE P 
GCSE 32.00 34.56 2.55 35.90 36.87 0.96 .004 
English 0.77 0.73 1.06 0.69 0.68 0.97 
GCSE 33.01 34.43 1.42 33.36 34.21 0.85 .254 
Maths 0.73 0.69 1.00 0.63 0.62 0.88 
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